The present invention relates to an information processing and display apparatus such as a personal computer and, more particularly, to a multimedia information processing apparatus.
A conventional multimedia information processing apparatus is an apparatus implemented by information processing apparatuses such as personal computers (hereinafter called PC's). The conventional multimedia information processing apparatus is capable of handling in an integral manner a plurality of information media. The plurality of information media include audio data, animated pictures and still images (simply called images), any of which may come from external sources.
A PC-based conventional multimedia information processing apparatus is configured generally by inserting a circuit board containing electrical circuits for performing a desired function into an extended input/output (I/O) bus slot of the PC. One such board is a scan converter board that converts NTSC (National Television System Committee) standard video signals (hereinafter called NTSC signals) output by video equipment such as a video camera into PC display video signals for display on a display unit of the PC. Another typical board is an audio signal processing board that processes audio signals for audio output via a speaker system.
When the multimedia information processing apparatus processes images, it is common practice to incorporate an image data compression/expansion board in the apparatus. This is because images include huge amounts of data to be processed.
Aside from the above-described boards that handle only one information medium per board, there exist boards each capable of integrally addressing a plurality of information media such as voice and images. Many boards that handle images may be supplemented functionally by an on-board display memory for enhancing the resolution of image display and for increasing the number of display colors available.
A conventional mode of display on PC's is multi-window display. Multi-window display is a manner of display in which the results of processing by a plurality of programs are shown separately in a plurality of rectangular areas (called windows) on the display screen, the positions and sizes of the areas being determined beforehand.
In the conventional multimedia information processing apparatus utilizing multi-window display, the size of each rectangular display area on the screen is set to be a full-screen, a half-screen, a quarter-screen, etc.
Described below are some known examples in which the display data of an image from an external video apparatus are superimposed on the display data of the PC. This process is known as keying. A first example of keying involves superimposing image data onto a specific color of PC display data. This method is called chromakey.
FIG. 17 is a schematic view of a device which implements a chromakey process. In this example, both the image data and the PC display data are in analog format as they are superimposed onto one another. In FIG. 17, reference numeral 1505 is the image data from a video apparatus, 1506 is the PC display data, 1509 is a display data switch for chromakey operation and 1510 is a display data switching signal that is input to the display data switch for switching control. 1502 is a color generator for generating data about the color on which to superimpose the image and 1503 is a comparator for comparing the color data from the color generator 1502 with the PC display data 1506 for color coincidence.
In the color generator 1502 is set beforehand the color on which to superimpose the image data. The display data switch 1509 is usually set to the position of the PC display data 1506 to allow the data 1506 to appear on display 123. The PC display data 1506, besides being fed to the display data switch 1509, is also input concurrently to the comparator 1503 for comparison with the color data generated by the color generator 1502. If a color coincidence is detected in the comparison, the comparator 1503 sends the display data switching signal 1510 to the display data switch 1509 so that the switch 1509 will be switched to the position of the image data 1505. This allows the image data 1505 to appear on the display 123.
Another example of keying will be described below. This example involves using a keying data memory (called a control plane) which is constructed to reflect the same display resolution as that of the PC and which offers at least one bit of storage capacity per pixel. The keying data is data whose values are used unchanged for the switching of display data.
FIG. 18 is a schematic view of a conventional chromakey process using a control plane. In FIG. 18, reference numeral 1501 is the control plane. The remaining components are the same as in FIG. 17. The data of the control plane 1501 is predetermined. For example, a "1" causes the image data 1505 to be displayed and a "0" causes the PC display data 1506 to be displayed. One's are written to those positions in the control plane 1501 which reflect those positions on the display 123 where images are to be displayed. The data of the control plane 1501 is read out concurrently with the data of a PC display data memory, not shown. The data that is read out are input in the form of a display data switching signal 1510 to the display data switch 1509 for and between image data 1505 and PC display data 1506.
An application of the multimedia system is a video conference system. An example of a conventional video conference system based on the multimedia system will now be described. The conventional video conference system is configured by connecting PC's or like terminals established at a plurality of locations via a network. These terminals are interconnected to let their users hold a video conference.
The conventional video conference system works typically as follows. Images coming from a plurality of terminals are composed at each terminal into a single composite picture containing the multiple input images, as disclosed in Japanese Patent Laid-Open No. 2-63288 or No. 2-63289.
FIG. 24 is a view of a display example generated by a conventional video conference system. The figure shows images displayed on a display unit of a terminal configured in the system, the images being sent from other 16 terminals. As indicated, one composite picture comprises images 2201 through 2216 showing waist-up images of 16 participants of the conference.
One aspect of the conventional multimedia system outlined above involves utilizing extension circuit boards having thereon various circuits to implement various functions. The scheme allows PC-based multimedia equipment to be established with ease.
However, there are some boards such as a scan converter board and an audio signal processing board each capable of addressing only one information medium. Each of these boards, lacking compatibility with other boards, is incapable of operating in synchronism therewith. For example, an audio signal processing board cannot provide its audio output in synchronism with images coming from a scan converter board. Where there arises a need to expand the functions of the existing resources to deal with a plurality of information media, the user has heretofore had no choice but to discard the existing resources and to purchase anew a board that would address such multiple information media.
A board capable of addressing a plurality of information media, e.g., a board dealing with video and audio data at the same time, may handle the multiple media in synchronism. However, where it is desired to supplement the board with other functions, the user is faced with the same restrictions as those applied to the above-mentioned scan converter and audio signal processing board. If the user has no need for some of the multiple functions provided by the single board, there is no choice for the user but to make the unnecessary expenditure to purchase the entire board.
If the display memory on the board is expanded to enhance the resolution of image display or to increase the number of display colors, another disadvantage is liable to occur. That is, the increase in the number of pixels or in the number of display colors leads to an increase in the amount of data per pixel. That in turn results in a significantly reduced speed at which to write data to the display memory for display purposes.
Another aspect of the conventional system involves displaying images from external video equipment within a window framework. In this case, the size of image display is controlled easily. However, this feature hampers the ability to change the window display size as desired.
The chromakey process of the conventional system has the advantage of requiring no specific control measures when a mouse cursor or other graphic image is superimposed on an image being displayed. However, if another window is superimposed on a multi-window image being displayed and if the superimposed window has the same color as that of the background image, the background image can become visible in the foreground with the window becoming transparent.
In cases where keying is performed using a control plane arrangement, the background image under a superimposed window does not become visible, with the window kept from becoming transparent. This is because the control plane data of the hidden image part are of values that select PC display data (e.g., 0's). However, if it is desired to display a mouse cursor or like graphic image superimposed on the existing image, the same shape as, say, the mouse cursor must be written to the control plane as needed. This turns out to be a considerable burden on the keying process.
Where the multimedia system is utilized in a video conference system, images from a plurality of sources are composed into a single composite image for display on each terminal. With this configuration, it is impossible to divide the composite image into original images for display at any desired positions on the display screen.